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Abstract. Despite the critical importance of Computer Assisted Language Learning 
(CALL) Teacher Education (CTE) programmes, L2 practitioners often report that 
there are not many such onsite/online programmes, and the ones that exist may not 
directly meet their specific needs. Moreover, there seems to be a lack of systematic 
evaluation studies of such courses. As a result, problems in such courses cannot be 
diagnosed, and they are left unresolved. This paper presents a study undertaken to 
evaluate an online Master’s of Arts (MA) in CALL programme in the Republic of 
Cyprus. The investigation was based on a conceptual multidimensional e-learning 
evaluation model rated by the students. Data were collected from 25 graduate 
students via an online anonymous survey. This focussed on the participants’ 
perceptions of the value of the following aspects: (1) their engagement; (2) the 
course and its organisation, teaching mode, and materials; (3) course strengths; (4) 
course aspects most helpful to learning; and (5) course aspects which were obstacles 
to learning. This paper discusses the findings and offers some first recommendations 
for further improvement to the MA in CALL programme. 
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1. Introduction 


In the last decades, substantial research has been dedicated to CTE. The research 
of Hubbard and Levy (2006), Torsani (2016), Hauck and Kurek (2017), and 
Jordano de la Torre (2019) are indicative. However, a brief review in the area of 
CTE programme evaluation reveals that this area still “warrants considerably more 
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professional — and international — attention” (Hubbard & Levy, 2006, p. ix). This 
paper presents a study undertaken to evaluate an online MA in CALL programme 
in the Republic of Cyprus. The investigation was based on a conceptual 
multidimensional e-learning evaluation model rated by the students. 


2. Method 


The present investigation concentrates on the evaluation of an online MA in CALL 
programme (running since September 2015) from the students’ viewpoint. The 
paper presents the results of the research study implemented at the end of the 
academic year 2018-2019. 


2.1. Participants 


Nineteen students (80% of the total programme population over a period of four 
years) from Cyprus University of Technology, ranging between 22 to 60 years old, 
participated in this study. The grade point average mean of females was 2.80, while 
that of males was 2.52, out of a possible 4.00. 


2.2. | Aconceptual multidimensional e-learning evaluation model 
The investigation proposed a conceptual multidimensional e-learning evaluation 
model focussing on: (1) their engagement; (2) the course and its organisation, 
teaching mode, and materials; (3) course strengths; (4) course aspects most helpful 
to learning; and (5) course aspects which were obstacles to learning 
2.3. Procedure 
The data collection was conducted between May and June 2019. The online survey 
was distributed to 25 students. Nineteen participants submitted a signed consent 
form and completed the online survey anonymously. 
2.4. _ Instrument and data collection 
The online survey contained questions on the eight modules: 

¢ LCES510 Second Language Acquisition; 


¢ LCES511 Teaching Methodologies; 
¢ LCES512 L2 Instructional Technologies; 


335 


Salomi Papadima-Sophocleous and Christina Nicole Giannikas 


¢ LCES513 L2 Curriculum Review and Improvement and Evaluation; 
¢ LCES514 Research Methodologies in Applied Linguistics; 

¢ LCES515 Computer Assisted Language Assessment and Testing; 

¢ LCES516 Mobile Assisted Language Learning; 

¢ LCES517 CALL from the Student Perspective. 


The survey consisted of 58 questions and supported quantitative and qualitative 
elements. Likert scale closed-ended questions were applied. Respondents were 
presented with a set of verbal answer options — Disagree (D), Strongly Disagree 
(SD), Neutral (N), Agree (A), Strongly Agree (SA) — to cover a range of opinions, 
and 32 open-ended questions. Open-ended questions were also included and 
covered three aspects: (1) course strengths; (2) course aspects most helpful to 
learning; and (3) course aspects which were obstacles to learning. 


3. Data analysis, results and discussion 


3.1. Quantitative data 


A key research issue was to investigate the merits and shortcomings of the MA 


course. Participants were asked to evaluate their engagement in the process 
(Table 1). 


Table 1. Students’ engagement 


= 

= S = a en + wn N=) bt 
= = = = = = = = 

Sp mak ef ae mal mai ai ff bal 

3 

op oy oy Fo icy Boy Fo oy icy 

=| 1S) 1S) 1S) 1S) 1S) 1S) 1S) 1S) 

si ae |2=e [AS (AS [AS [AE |AS [AS 

Ql. Effort n:IN n:INI n:IN n:1N n:1N 
n:l0OA | n:6A n:5A n:6A nl0OA |n:6A n:8A n:8A 
n:9SA_ |n:12SA |n:13SA |n:13SA | n:9SA | n:12SA |n:1ISA (n:1ISA 


Q2. n:ISD 
Punctuality | :2D n:1D n:2D n:2D n:2D n:3D n:33D n:2D 
n:5N n:3N n:3N n:3N n:3N n:4N n:4N n:5N 
n:3A n:6A n:6A n:5A n:7A n:4A n3A n:5A 
n:9SA |n:8SA |n:8SA |n:9SA |n:7SA /n:8SA |n:9SA  (n:7SA 
Q3. n:1D n:1D 
Participation n:3A jn:5A jn 5A im 5A [ms 3A om3A |m3A om 5A 
consistency |n: 16SA |n: 14SA | n: 14SA | n: 14SA_ nn: 16SA | n: ISSA) n: ISSA | n: 14SA 
(webinars) 


336 


Time to evaluate: the students’ perspective of an online MA in CALL programme 
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High numbers show that students believed to have put in a considerable amount of 
effort (Q1), and are highly consistent when it comes to attending webinars (Q3). 
However, numbers tend to drop in Q2 and Q4, regarding students’ punctuality and 
consistency in completing their weekly tasks. While students feel they make great 
efforts to evolve as students and are present when there is immediate contact with 
their peers and instructors during online webinars, maintaining consistency and 
punctuality in their autonomous learning proved a challenge. 


The study investigated structure and materials used in order to evaluate the quality 
of the course and its content. Table 2 displays data per module. 


Table 2. Course structure and material 
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The students’ evaluations of the structure and material used throughout the MA 
programme offered insights on whether the planning and construction of the 
modules are well-presented, efficient, and beneficial to students’ learning. The 
responses complement the previous segment of student engagement, as the 
structure and materials used in each module, to an extent, explain the positive 
outcomes of Table |. In the current segment, the only section where students felt 
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indifferent was in Q4, where they were asked about their assignments. This outcome 
was anticipated after the analysis of the first segment, as the students’ greatest 
challenge was the consistency and punctuality of their assignments. This comes 
back to the nature of the online MA and the fact that students are encouraged to 
work autonomously and construct their learning. Their autonomous skills demand 
greater development, as students need further practice in self-training to reach the 
peak of their productivity. 


3.2. Qualitative data 


The qualitative component uncovered students’ thoughts and opinions. The open- 
ended questions focussed on (1) strengths of the course, (2) parts that aided students’ 
learning, and (3) parts that were obstacles to their learning of each module. Themes 
re-occurred for all modules and emerged from the data coding. The most common 
themes are described in Table 3 below. 


Table 3. Students’ thoughts and opinions 


Course strength 


Parts of the programme 
that aided students’ learning 


Parts that were obstacles 
to their learning 


Flexibility Guidance No obstacles (6 students) 
Clarity Collaboration Workload (4) 
Constructivism Computer-based tools Meeting deadlines (3) 
Reflection Weekly activities Continuous reflective 


journals (1) 


Increased digital Reflections Weekly reflection exhausted 

awareness me sometimes (1) 

Instant feedback Resources Sometimes feedback was 
not given promptly or it 
was given not long before 
submission deadlines (1) 

Relatable to daily Webinars 

practice 

CALL-focus 


Combination of 
theory and practice 


The students found the postgraduate programme materials, webinars, guidance, 
and content highly beneficial and they have advanced their CALL understanding 
and practice. Six stated they did not identify any parts of the programme hindering 
their learning. However, four found the workload heavy; three found meeting 
deadlines difficult. One found continuous reflective journal writing tiring; another 
found it sometimes exhausting. One also had an issue with feedback promptness. 
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One interesting outcome is that all modules have been similarly evaluated sharing 
the majority of the same positives and challenges. 


4. Conclusions 


There is a need to identify the extent to which programmes meet students’ needs. 
The present paper reflects the effort to evaluate the MA in CALL from the students’ 
perspective. The results were mostly positive, both in terms of student input and 
in terms of the programme evaluation. However, further investigation into the 
workload, weekly activities, and time limit issues are necessary. 
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